1. Field of the Invention
The invention relates to problem area location in an image signal, and more specifically, to occlusion detection and halo reduction in motion compensated pictures.
2. Description of the Related Art
Every motion compensated scan-rate conversion method is confronted with the problem of occlusions in a sequence. Several approaches have been attempted to cope with it. In many cases, the effort has been devoted at improving the quality of the motion estimation method in order to have very precise motion boundaries (e.g., See Ref. 1). But in the regions where covering or uncovering occurs, and where the motion estimation is performed by analyzing two successive frames, motion estimation is an ill-posed problem (see Ref. 2) and cannot yield good results. To overcome this problem, many authors propose to use three frames (see Refs. 3, 4, and 5) or four frames (see Ref. 6), for both motion estimation and motion compensation. When architectural constraints suggest to use two frames only, an ad hoc interpolation strategy has to be introduced. This strategy can be applied on every pixel of the image or can be preceded by the localization of critical areas, i.e., by a segmentation of the image.
In Ref. 7, a method was disclosed for motion compensated picture signal interpolation this reduces the negative effect of covering and uncovering on the quality of interpolated images. In the described case, that applies an order statistical filter in the up-conversion to replace the common MC-averaging, interpolated pictures result from pixels taken from both adjacent fields.
In Ref. 2 and in Ref. 8, a segmentation for the same purpose was described. This segmentation is based on a motion detector, and can only produce reliable results if covering and uncovering occur of stationary backgrounds.
In Ref. 9, a method was disclosed that allows a reduction of halo defects in architectures that enable access to one field only, or in systems particularly designed to have access to one field only in order, to obtain the increased resolution of an interpolation according to Ref. 10.
In Ref. 11, a method was disclosed that uses two motion estimators, a causal motion estimator (that predicts the future from the past) and an anti-causal motion estimator (that predicts the past from the future). Depending on which one of the two estimators gives the xe2x80x98best matchxe2x80x99, the area is classified as covered or uncovered, and the corresponding luminance value is taken from the previous or the next field.
In Ref. 12, the interpolation strategy is tuned depending on the xe2x80x98difficultiesxe2x80x99 of the image part. It combines several of the well-known algorithms for motion compensation, aiming at exploiting their complementary strengths. The task of selecting the appropriate algorithm is assigned to an Ordered Statistical filter. Where no adequate strategy is available, like in covered/uncovered areas, it aims at softening the resulting artifacts.
Instead, in Ref. 13, it is stated that the general rule for an effective interpolation seems to be: xe2x80x9cif it is not possible to shift a small detail correctly because of faulty motion vectors, better suppress it than smooth itxe2x80x9d. This is achieved, when there is a faulty vector assigned and a correlated picture content, extending the median mask used to filter the candidates from the neighboring frames, and where there is no correlated picture content, using the probability distribution function of a Centered Median Filter, to select the candidates.
It is, inter alia, an object of the invention to provide a straightforward and reliable occlusion detection and halo reduction. To this end, a first aspect of the invention provides a method and device for locating problem areas in an image signal. A second aspect of the invention provides a corresponding method and device for interpolating images between existing images. A third aspect of the invention provides an image display apparatus incorporating such an image interpolation device.
In a method of locating problem areas in an image signal, a motion vector field is estimated for the image signal, and edges are detected in the motion vector field. In a corresponding method of interpolating images between existing images, image parts are interpolated in dependence upon a presence of edges; preferably, an order statistical filtering is used at edges.
The current invention basically adapts the interpolation strategy depending on a segmentation of the image in various areas. Contrary to Refs. 2and 8, the current invention aims to be valid even if both foreground and background are moving.
These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.